Evaluating the articulation of programme theory in practice as observed in Quality Improvement initiatives

Background: The Action-Effect Method(AEM) was co-developed by NIHR CLAHRC Northwest London (CLAHRC NWL) researchers and QI practitioners, building on Driver Diagrams(DD). This study aimed to determine AEM effectiveness in terms of technical aspects (how diagrams produced in practice compared with theoretical ideals) and social aspects (how engagement with the method related to social benefits). Methods Diagrams were scored on criteria developed on theoretical ideals of programme theory. 65 programme theory diagrams were reviewed (21 published Driver Diagrams (External DDs), 22 CLAHRC NWL Driver Diagrams (Internal DDs), and 21 CLAHRC NWL Action-Effect Diagrams(AEDs)). Social functions were studied through ethnographic observation of frontline QI teams in AEM sessions facilitated by QI experts. Qualitative analysis used inductive and deductive coding. Results ANOVA indicated the AEM significantly improved the quality of programme theory diagrams over Internal and External DDs on an average of 5 criteria from an 8-point assessment. Articulated aims were more likely to be patient-focused and high-level in AEDs than DDs. The cause/effect relationships from intervention to overall aim also tended to be clearer and were more likely than DDs to contain appropriate measure concepts. Using the AEM also served several social functions such as facilitating dialogue among multidisciplinary teams, and encouraging teams to act scientifically and pragmatically about planning and measuring QI interventions. Implications: The Action-Effect Method developed by CLAHRC NWL resulted in improvements over Driver Diagrams in articulating programme theory, which has wide-ranging benefits to quality improvement, including encouraging broad multi-disciplinary buy-in to clear aims and pre-planning a rigorous evaluation strategy

BEYOND CLINICAL GUIDELINES: HOW CARE PATHWAYS AND QUALITY-IMPROVEMENT METHODS CAN SUPPORT BETTER ALLERGY CARE

The increasing prevalence of allergic disease has resulted in the recognition of allergy as a global public health concern. Yet health services worldwide appear to be ill-equipped to deliver high-quality allergy care. Clinical guidelines have been developed to describe what high-quality care looks like for most allergic diseases. However, allergy guidelines do not describe how the delivery of such care is organised across clinicians and provider organisations with varying degrees of access to allergy expertise and clinical resources. In this article, we describe how care pathways can be used to improve the organisation and delivery of allergy care in accordance with the characteristics of allergic disease and local constraints in the health service. We then describe how quality-improvement methods can support the successful realisation of allergy care pathways in practice. Realising care pathways involves a highly complex process of changing the way care is practised and organised. This could involve developing a new service, clinical training or other interventions. Qualityimprovement methods were developed as a guide to navigate and support the process of change and improvement

Simple rules for evidence translation in complex systems: a qualitative study

Background Ensuring patients benefit from the latest medical and technical advances remains a major challenge, with rational-linear and reductionist approaches to translating evidence into practice proving inefficient and ineffective. Complexity thinking, which emphasises interconnectedness and unpredictability, offers insights to inform evidence translation theories and strategies. Drawing on detailed insights into complex micro-systems, this research aimed to advance empirical and theoretical understanding of the reality of making and sustaining improvements in complex healthcare systems. Methods Using analytical auto-ethnography, including documentary analysis and literature review, we assimilated learning from 5 years of observation of 22 evidence translation projects (UK). We used a grounded theory approach to develop substantive theory and a conceptual framework. Results were interpreted using complexity theory and ‘simple rules’ were identified reflecting the practical strategies that enhanced project progress. Results The framework for Successful Healthcare Improvement From Translating Evidence in complex systems (SHIFT-Evidence) positions the challenge of evidence translation within the dynamic context of the health system. SHIFT-Evidence is summarised by three strategic principles, namely (1) ‘act scientifically and pragmatically’ – knowledge of existing evidence needs to be combined with knowledge of the unique initial conditions of a system, and interventions need to adapt as the complex system responds and learning emerges about unpredictable effects; (2) ‘embrace complexity’ – evidence-based interventions only work if related practices and processes of care within the complex system are functional, and evidence-translation efforts need to identify and address any problems with usual care, recognising that this typically includes a range of interdependent parts of the system; and (3) ‘engage and empower’ – evidence translation and system navigation requires commitment and insights from staff and patients with experience of the local system, and changes need to align with their motivations and concerns. Twelve associated ‘simple rules’ are presented to provide actionable guidance to support evidence translation and improvement in complex systems. Conclusion By recognising how agency, interconnectedness and unpredictability influences evidence translation in complex systems, SHIFT-Evidence provides a tool to guide practice and research. The ‘simple rules’ have potential to provide a common platform for academics, practitioners, patients and policymakers to collaborate when intervening to achieve improvements in healthcare

Criteria for evaluating programme theory diagrams in quality improvement initiatives: a structured method for appraisal

Background: Despite criticisms that many Quality Improvement (QI) initiatives fail due to incomplete programme theory, there is no defined way to evaluate how programme theory has been articulated. The objective of this research was to develop, and assess the usability and reliability of, scoring criteria to evaluate programme theory diagrams. Methods: Criteria development was informed by published literature and QI experts. Inter-rater reliability was tested between two evaluators. 63 programme theory diagrams (42 driver diagrams and 21 action effect diagrams) were reviewed to establish whether the criteria could support comparative analysis of different approaches to constructing diagrams. Results: Components of the scoring criteria include: assessment of overall aim, logical overview, clarity of components, cause/effect relationships, evidence, and measurement. Independent reviewers had 78% inter-rater reliability. Scoring enabled direct comparison of different approaches to developing programme theory; Action-Effect diagrams were found to have had a statistically significant but moderate improvement in programme theory quality over Driver Diagrams; no significant differences were observed based on the setting in which Driver Diagrams were developed. Conclusions: The scoring criteria summarise the necessary components of programme theory that are thought to contribute to successful QI projects. The viability of the scoring criteria for practical application was demonstrated. Future uses include assessment of individual programme theory diagrams, and comparison of different approaches (e.g. methodological, teaching or other QI support) to produce programme theory. The criteria can be used as a tool to guide the production of better programme theory diagrams, and also highlights where additional support for QI teams could be needed

Survey of patient and public perceptions of electronic health records for healthcare, policy and research: Study protocol

BACKGROUND: Immediate access to patients’ complete health records via electronic databases could improve healthcare and facilitate health research. However, the possible benefits of a national electronic health records (EHR) system must be balanced against public concerns about data security and personal privacy. Successful development of EHR requires better understanding of the views of the public and those most affected by EHR: users of the National Health Service. This study aims to explore the correlation between personal healthcare experience (including number of healthcare contacts and number and type of longer term conditions) and views relating to development of EHR for healthcare, health services planning and policy and health research. METHODS/DESIGN: A multi-site cross-sectional self-complete questionnaire designed and piloted for use in waiting rooms was administered to patients from randomly selected outpatients’ clinics at a university teaching hospital (431 beds) and general practice surgeries from the four primary care trusts within the catchment area of the hospital. All patients entering the selected outpatients clinics and general practice surgeries were invited to take part in the survey during August-September 2011. Statistical analyses will be conducted using descriptive techniques to present respondents’ overall views about electronic health records and logistic regression to explore associations between these views and participants’ personal circumstances, experiences, sociodemographics and more specific views about electronic health records. DISCUSSION: The study design and implementation were successful, resulting in unusually high response rates and overall recruitment (85.5%, 5336 responses). Rates for face-to-face recruitment in previous work are variable, but typically lower (mean 76.7%, SD 20). We discuss details of how we collected the data to provide insight into how we obtained this unusually high response rate

Adapting improvements to context: when, why and how?

There is evidence that practitioners applying quality improvements often adapt the improvement method or the change they are implementing, either unknowingly, or intentionally to fit their service or situation. This has been observed especially in programs seeking to spread or ‘scale up’ an improvement change to other services. Sometimes their adaptations result in improved outcomes, sometimes they do not, and sometimes they do not have data make this assessment or to describe the adaptation. The purpose of this paper is to summarize key points about adaptation and context discussed at the Salzburg Global Seminar in order to help improvers judge when and how to adapt an improvement change. It aims also to encourage more research into such adaptations to develop our understanding of the when, why and how of effective adaptation and to provide more research informed guidance to improvers. The paper gives examples to illustrate key issues in adaptation and to consider more systematic and purposeful adaptation of improvements so as to increase the chances of achieving improvements in different settings for different participants. We describe methods for assessing whether adaptation is necessary or likely to reduce the effectiveness of an improvement intervention, which adaptations might be required, and methods for collecting data to assess whether the adaptations are successful. We also note areas where research is most needed in order to enable more effective scale up of quality improvements changes and wider take up and use of the methods